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STATEMENT REGARDING FEDERALLY 
SPONSORED RESEARCH OR DEVELOPMENT 
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FIELD OF THE INVENTION 

[09] The present invention is related to a method and implementation of enhancing a 
facial recognition process to increase the judgment confidence on identifying a person 
firom a large image database of multiple persons. 

BACKGROUND OF THE INVENTION 

[10] In traditional facial identification processes of identifying one person firom an 
image database of multiple persons in 2D facial recognition, a face photo of the person is 
compared (or called "^matching") with all face photos in the database using a recognition 
algorithm or recognition software. The 2D facial recognition algorithm or software will 
generate a confidence percentage for each matching photo in the database on how much 
similarity the 2D facial recognition algorithm finds between the face photo of the person 
and the matching photo in the database. The match of the highest confidence percentage 
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or several matches with top confidence percentages are returned for the operator to make 

a judgment as to whether there are matches. 

f ll] Traditional 2D facial recognition algorithms, however, normally have problems 
identifying face photos because of a large number of variables, such as the lighting 
present at the time a particular photo is taken or the angle at which the photo is taken, 
with respect to the face photo being taken of the person and the face photos in the 
database. Other such variables would be apparent to those skilled in the art. These 
variables can cause the confidence percentages generated by the 2d facial recognition 
algorithm or software to be artificially low. 



SUMMARY OF THE INVENTION 

[12] The present uivention enhances the matching process by reconstructing an image 
database of multiple persons into one in which each person has multiple pictures 
representing different angles, lighting conditions and etc. Preferable, each person will 
have a similar or identical number of photos in the reconstructed database. The invention 
enhances the confidence percentage for a particular match based upon the number of 
matching photos found for a particular person. The present invention provides a metiiod 
and implementation on how to construct and define such increased confidence over 
existing single-image-per-person facial recognition systems. 

[13] Still other aspects, features, and advantages of the present invention are readily 
apparent from the following detailed description, simply by illustrating preferable 
embodiments and implementations. The present invention is also capable of other and 
different embodiments, and its several details can be modified in various respects, all 
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without departing from the spirit and scope of the present invention. Accordingly, the 
drawings and descriptions are to be regarded as illustration in nature, and not as 
restrictive. 

BRIEF DESCRITION OF THE DRAWINGS 

[14] The accompanying drawings, which are incorporated in and constitute a part of 
this specification illustrate some embodiments of the invention and, together with the 
description, serve to explain the objects, advantages, and principles of the invention. In 
the drawings, 

[15] FIG. 1 illustrates a conceptual dataflow diagram of the enhanced facial 
recognition architecture in accordance with the present invention. 

DESCRIPTION OF THE PREFERRED EMBODIMENT 

[16] The enhanced facial recognition system and method of the present invention is 
built upon existing 2D facial recognition algorithms. The present invention relates to 
enhancement of a confidence level of an identity of a person from an image database of 
multiple persons, sometimes called a one-to-many identification process. 

[17] Referring to FIG. 1, normally each person in a 2D image database 101 has only 
one (frontal) or two (frontal and profile, i.e. 90 degree) 2D pictures in the database. Given 
such a limited database, a first step of the present invention is to reconstruct 109 the 2D 
image database 102 in such a way that each person will have same nimiber or similar 
number (preferably the difference in number being less than ±5% of the average number 
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[18] There exists a technology that can generate a 3D module from a 2D frontal picture 
or from both a 2D frontal picture and a 2D profile picture of a person (Such as the 
software from CyberExtruder, http://www.cyberextruder.com/). This 2D to 3D 
processing is sometimes referred to as "2D-3D processing." The 3D module can be 
rotated and be added with different lighting and other parameters as one wishes. Such 
lighting parameters may include brightness, hue, contrast, the angle of lighting, and 
others. One also can save a 2D picture (sometimes called "parsed image" or "morphed 
image") from the generated 3D module at each orientation and lighting condition from 
the 3D module. Theoretically, one could generate a virtually infinite number of 2D 
pictures of a person with different orientation, lighting and other parameters. This 3D to 
2D processing may be referred to as "3D-2D processing," "3D parsed to 2D processing," 
or "parsing a 3D unage into multiple 2D images." The whole process of converting one 
or two 2D picture(s) of a person into a 3D module and then fiirther generating multiple 
parsed 2D pictures of a person is somethnes referred as "2D-3D-2D processing." While 
the above represents one method of generating multiple pictures of a person for the 
reconstruction of a 2D image database, other methods may be used in connection with the 
present invention. 

[19] Referring to FIG. 1, using the above 2D-3D-2D processing technology, one can 
generate multiple 2D pictures with different orientations, different lighting conditions and 
other parameters from one or two 2D pictures of a person. Typically based on intended 
applications, one original image is to be parsed into 50 to 1000 parsed images. This 
provides a method 109 to reconstruct an original 2D image database 101 into an 
enhanced 2D unage database 102. Further, rules may be set for the reconstruction. First, 
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the number of pictures generated for each person preferably is set to be substantially 
same (or the difference in number is less than ±5% of the average number of pictures per 
person). Second, the orientation, lighting conditions and other parameter settings 
preferably are set to be substantially the same for all persons in the image database. 
Third, one original picture per person, /.e., the picture from which the multiple parsed 
images was generated and normally is a frontal picture, is included in the reconstructed 
database. These three rules provide for each person in the enhanced 2D image database 
102 to be relatively equivalent. Other reconstruction rules, or a subset of these three 
rules, are, of course, possible and may be used with the present invention. 

[20] Still referring to FIG. 1, in a 2D image database having multiple pictures of each 
of multiple people (such as the reconstructed database), each image in the 2D image 
database 102 may be digitized into a template to be used by a 2D facial recognition 
matching algorithm for one-to-one comparison with a photo taken of a person. At this 
point, each person in the enhanced image database 102 has multiple parsed images 
(including original image) and multiple templates (one template per each parsed image 
and the original image). Since only templates are used in 2D facial matching algorithm, 
and there is no need to display a large number of images of the same person to an 
operator when matches are found, all but one or a few of the multiple parsed images can 
optionally be removed after their templates are generated in order to save storage space. 
Typically, the original frontal image would be kept for display to an operator in the event 
of a match for a particular person in the database. Each person in the enhanced 2D image 
database 102 has one group of templates, and every template in a person's group is 
assigned with same group ID. 
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[21] Referring to FIG.l, the present invention shows an enhanced judgment process 
(105, 104, 106, 107, and 108) based on newly reconstructed 2D image database 102. 

[22] Still referring to FIG. 1, the process starts from one incoming target unage 103, 
and the goal is to find matches which have above a pre-selected confidence threshold, say 
PSl in percentage (%), from the enhanced 2D unage database 102. 

[23] Still referring to FIG. 1, the present process starts by finding 105 matches of the 
incoming target image 103 from the enhanced 2D image database 102, and the matches 
should have over a second pre-selected confidence threshold, say PS2 in percentage (%) 
(PS2 is also called "qualification percentage"), and choose a number N2 (called 
"qualification candidate nimiber") of top matches from the matches to form a group, 
called voting group 104. And typically choose PS2 <= PSl, and N2 ideally be no limit. 
If N2 being unlimited results in too many matches and therefore a slow speed, N2 is 
typically chosen to be the number of morphed images each person has, which is in 100s 
or 1000s. In the well-constrained environments in which the incoming target unage is 
known to be within certain angle and lighting conditions, then the niunber of morphed 
images can be configured to be much less. 

[24] Still referring to FIG. 1, it is possible that one person has multiple matches in the 
voting group 104, and it is often the case in real world. And normally each match can 
have different matching confidence percentage. 

[25] Another possibility is no match above PS2. Therefore, there will be no match 
above PSl (>= PS2) either, and the final judgment or result is no match in such a case. 
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procedure is that if multiple judges say he is the target, it will be more confident than one 
judge says he is the target. 

[27] Still referring to FIG, 1, based on the above design philosophy, there are many 
ways to unplement the procedure 106 (called voting procedure). Here shows one possible 
way. Let us assume that there are A'^ number of matches to the incoming target in the 
voting group 104, and those matches correspond to L number of persons, each person 
has, respectively, Pi, /i2, Pi Pl number of matches, and pi'^p2-^P3'^ **^'^Pl=N, and 
L <= and each person has its maximum match value, respectively, ml, m2, m3, mL 
in the voting group 104. And choose a percentage number X that is the maximum extra 
confidence percentage added above the regular confidence percentage number from the 
2D facial matching algorithm. X is typically chosen to be 10 to 20 (%). Then each person 
in the votmg group 104 will have the final match confidence percentage, respectively, 
(mi + pi/N*X), (m2 + py3V*X), ... (mt + pi/N'^X). Another way to determine the final 
match confidence percentage is to choose the maximum value of each person in the 
voting group mi, ml, m3, ... mL, respectively, as the final match confidence percentage 
for each person in the voting group. If the final match percentage value is over 100, set 
to be 100. By the way, the match percentage value is between 0 and 100 including 0 and 
100. 

[28] Still referring to FIG. 1, then each person in the votmg group 104 will have one 
single fmal match percentage value. Choose those persons whose fmal match percentage 
value is over PS2 for final output 107, 108 if any, and display the result in sorted order. 

[29] The foregoing description of the preferred embodiments of the invention has been 
presented for purposes of illustration and description. It is not intended to be exhaustive 



8 



are possible in light of the above teachings or may be acquired from practice of the 
invention. The embodiments were chosen and described in order to explain the principles 
of the invention and its practical application to enable one skilled in the art to utilize the 
invention in various embodiments as are suited to the particular use contemplated. It is 
intended that the scope of the invention be defined by the claims appended hereto, and 
their equivalents. The entirety of each of the aforementioned documents is incorporated 
by reference herein. 
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